Lecture 5 slides: Point estimators

Let X be our data. Let θ̂ = T (X) be an estimator where T is some function. We say that θ̂ is unbiased for θ if Eθ[T (X)] = θ for all possible values of θ where Eθ denotes the expectation when θ is the true parameter value. Thus, the concept of unbiasness means that we are on average right. The bias of θ̂ is de ned by Bias(θ̂) = Eθ[θ̂] − θ. Thus, θ̂ is unbiased if and only if its bias equals 0. For example, if X is a random sample X1, ..., Xn from some distribution with mean μ and variance σ2, then, as we have already seen, E[μ̂] = μ and E[s] = σ2. Thus, sample average and sample variance are unbiased estimators of population mean and population variance correspondingly. There are some cases when unbiased estimators do not exist. As an example, let X1, ..., Xn be a random sample from a Bernoulli(p) distribution. Suppose that our parameter of interest θ = 1/p. Let θ̂ = T (X) be some estimator. Then E[θ̂] = ∑ (x ,...,x ) 0,1 n T (x1, ..., xn 1 n )P{(X1, ..., Xn) = (x1, ...xn)}. We know that ∈{ } for any (x1, ..., xn) ∈ {0, 1}n, P{(X1, ..., Xn) = (x1, ...xn)} = p i(1 − p) (1−xi) which is a polynomial of degree n p ∑ ∑